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Appendix A. The Louisiana Believe and Prepare pilot program 


This study examined early career outcomes for participants in the Louisiana Believe and Prepare pilot program 
that launched in 2014. Louisiana lacks enough qualified teachers in virtually every subject, with more acute 
shortages in math, science, and special education (Louisiana Department of Education, 2015; U.S. Department 
of Education, 2017). In 2014 the Louisiana Department of Education surveyed new teachers, district 
administrators, and teacher preparation faculty to determine how to prepare high-performing teachers, as 
defined by the Louisiana Compass teacher evaluation system (Louisiana Department of Education, 2014). The 
survey results highlighted the need for greater alignment and collaboration between school districts and teacher 
preparation programs to improve teacher preparation of teacher candidates, respond to district needs and 
teacher shortages, improve teacher hiring practices, and pair new teachers with qualified mentors and coaches. 


To respond to these challenges and needs, the Louisiana Department of Education launched the Believe and 
Prepare pilot program, which provided grants to school districts (known as “school systems” in Louisiana), in 
partnership with teacher preparation providers, to develop and implement teacher residency programs. The 
Believe and Prepare pilot program emphasized three components: competency-based coursework, a yearlong 
residency, and mentor teachers (table A1). The pilot was an opportunity for teacher preparation providers to 
develop a residency for practice-based preparation experiences to improve teacher preparation, recruitment, 
and retention. 


The pilot program was based on research that suggests that well designed and well implemented teacher 
residency models can improve teacher retention (Singh, 2017; Van Overschelde et al., 2017; Whipp & Geronime, 
2017). Researchers and advocates have emphasized the importance of including high-quality, long-term clinical 
experiences in the preparation of new teachers (Carnegie Corp, 1986; Darling-Hammond, 2006; Goodlad, 1991; 
Greenhill, 2010; Holmes Group, 1986; Levine, 2006; National Commission on Teaching and America’s Future, 
1996; National Council for Accreditation of Teacher Education, 2010). Teacher residencies are an increasingly 
popular strategy for providing clinical experience (Britt et al., 2016; Guha et al., 2017). Studies of teacher 
residency programs in New York City and Boston found higher retention rates within the district four or five years 
into teaching for new teachers prepared through residency programs in the district than for other new teachers 
(Papay et al., 2012; Sloan & Blazevski, 2015). A study of 12 teaching residency programs funded through U.S. 
Department of Education's Teacher Quality Partnership grants found similar rates of retention within schools for 
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new teachers trained through residency programs and those who were not but higher within-district retention 
rates for teachers trained through residency programs (Silva et al., 2015). 


Table A1. Believe and Prepare pilot program expected components 


Component Description 
Competency-based e Redesign teacher preparation coursework through a partnership between teacher preparation 
coursework program faculty and district leadership to emphasize essential teaching knowledge and skills and 


school-based, practice-oriented experience. 


e Develop learning outcomes, learning tasks, and performance assessments for preservice teachers 
through a partnership between district leaders and teacher preparation program faculty. 


e Use performance assessments of preservice teachers to determine their progress through the program. 


e Require preservice teachers to complete the revised coursework as a condition of receiving their 
degree 


Teacher residency e Implement a yearlong residency for preservice teachers to co-teach in the classroom up to five days a 
week during preservice teachers’ residency year (typically in year 4 of the degree program following 
the completion of other coursework). 


e Place preservice teachers in traditional public schools, based on staffing needs and local decisions; 
emphasize placement in rural schools and schools needing special education teachers. 

e Provide participating preservice teachers with a stipend of $2,000 from the Louisiana Department of 
Education during the residency year. 


Mentor teacher e Select mentors and teacher preparation program leads based on their demonstrated success with 
recruitment, selection, students and exhibited leadership qualities and skills. 
and training © Deliver teacher preparation program—based mentor training throughout the school year to develop 


and refine mentor coaching and feedback skills. 

e Provide participating mentor teachers with a stipend of $1,000 from the Louisiana Department of 
Education. 

e Provide opportunities (not requirements) for districts to develop teacher leadership pathways for 
effective teachers to engage in mentoring and coaching. 


Source: Authors’ compilation based on information available on the Louisiana Department of Education website. 


Through the Believe and Prepare pilot program, teacher preparation providers and their partner districts 
redesigned teacher preparation coursework to emphasize essential teaching knowledge and skills and school- 
based, practice-oriented experience. In addition, Believe and Prepare pilot teacher preparation providers and 
districts placed preservice teachers in residency at participating schools as part of their teacher preparation 
experience, a departure from typical practice in Louisiana in which preservice teachers completed one semester 
of student teaching to fulfill requirements for certification. In the residency period preservice teachers led 
classroom instruction most of the time, with support from mentors. Believe and Prepare partners were allowed 
autonomy to design coursework and preparation experiences to fit local needs and were expected to adjust 
preparation experiences to the needs of preservice teachers. The Louisiana Department of Education expected 
local variation in program implementation but did not systematically collect information on local program 
decisions and implementation for the pilot programs. 


Since 2014 the Louisiana Board of Elementary and Secondary Education has awarded more than $9 million in 
Believe and Prepare grants to teacher preparation providers and their district partners (Louisiana Department of 
Education, 2017). Three cohorts of teacher candidates participated in the Believe and Prepare pilot program: 
cohort 1 in 2014/15, cohort 2 in 2015/16, and cohort 3 in 2015/16 and 2016/17.? Districts could participate in 
multiple cohorts. To apply for the Believe and Prepare pilot grants, a district and its collaborating teacher 
preparation provider(s) submitted an application that detailed their plan for school-based teacher preparation. 


1 The grant award period for cohort 3 extended from 2015/16 through 2016/17. Some participants started in spring 2016, and others 
started in fall 2016. 
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The application was evaluated on four criteria: demonstration of needs, program design, program evaluation 
plan, and program sustainability and expansion plan. Districts that participated in at least one of the pilot cohorts 
are shown in figure A1. During the pilot years school districts and teacher preparation providers applied for the 
grants through a joint application; a teacher preparation provider could apply with multiple district partners. The 
Louisiana Department of Education oversaw the grants and required that teacher preparation providers identify 
the residency and placement sites and that partner districts confirm the residency placement. 


In 2016 the Louisiana Board of Elementary and Secondary Education, with support from the Louisiana Board of 
Regents, adopted regulations to expand the residencies statewide. The Louisiana Department of Education set 
the goal that all teacher preparation programs should include a yearlong residency as well as a competency- 
based curriculum and an expert mentor by July 2018 (https://www.louisianabelieves.com/teaching/believe-and- 
prepare). Candidates admitted into teacher preparation programs in the 2018/19 school year were the first 
statewide cohort to experience the required year-long residency and new competency-based curricula. 


Figure A1. Districts participating in Louisiana’s Believe and Prepare program pilot, 2014/15—2016/17 
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Source: Authors’ compilation based on data provided by the Louisiana Department of Education. 
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Appendix B. Data and methods 


This appendix describes the data sources, data preparation, study samples, and methods used to complete the 
analyses. 


Data sources 


This study used a combination of administrative data collected by the Louisiana Department of Education and 
publicly available data. 


The Louisiana Department of Education provided three datasets: 


1. Records of Believe and Prepare pilot participants who participated in the program from 2014/15 through 
2016/17. 


2. Certification data from the Teacher Certification Management System from 2014/15 through 2017/18 
(certification dataset). 


3. Teaching records from the Profile of Educational Personnel from 2014/2015 through 2018/19 
(employment dataset) 


The records could be linked across the three datasets using a random unique identifier for each teacher, when 
available. 


The study also employed publicly available data from the Louisiana Department of Education’s Data Center 
webpage (https://www.louisianabelieves.com/resources/library/data-center) and the National Center for 
Education Statistics Common Core of Data from 2014/15 through 2017/18 (National Center for Education 
Statistics, 2016, 2017, 2018, 2019). 


Data preparation 


Records of Believe and Prepare pilot participants. The dataset consisted of 744 records of Believe and Prepare 
pilot participants, based on records provided by grantee teacher preparation providers and school districts to the 
Louisiana Department of Education. Variables included unique random individual identification number, cohort, 
teacher preparation program name, teacher preparation subject, undergraduate major, district of residency, 
school of residency, teacher preparation program year, and teacher preparation program completion status. 


The random unique identifiers were assigned by the Louisiana Department of Education. Of the 744 records of 
pilot program participants, 491 (66 percent) could be matched to the data collected through the Profile of 
Educational Personnel End of Year data submission that district data managers completed for 2014/15-2016/17. 
These participants were assigned random unique identifiers. The remaining 253 participants (34 percent) could 
not be matched to the department’s personnel records and so were not assigned unique identifiers. 


The Louisiana Department of Education used the data submitted by teacher preparation program providers and 
district participants for Believe and Prepare funding to match to data collected through the Profile of Educational 
Personnel End of Year data submission that district data managers completed to validate and assign social security 
numbers to program participants. End of Year records included data from 2014/2015 through 2016/17. Only 21 
percent of Believe and Prepare program records were reported with Social Security numbers. The Louisiana 
Department of Education used participants’ first and last name, district code, and school code to match records 
and validate Social Security numbers in the program records or assign correct Social Security numbers (from the 
End of Year data) to participants. The department conducted automated matches based on those variables as well 
as manual matches to look up records for matches and was able to validate or assign Social Security numbers to 
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66 percent of program participants. Random unique identifiers were then generated for these participants. 
Believe and Prepare pilot participants with identifiers and those without identifiers differed from each other in 
cohort and teacher preparation program subject (see tables C12 and C13 in appendix C). However, there were no 
statistically significant differences in the characteristics of the school or district of residency between the two 
groups (see tables C14 and C15 in appendix C). 


The study team performed the following key data preparation steps: 


e Requested that the Louisiana Department of Education identify and remove duplicate records among the 253 
records that could not be matched to the department's certification or personnel records. That reduced the 
number to 245 unique records without identifiers. Because these 245 records could not be linked to 
certification or personnel records, they could not be used to examine the outcomes of interest. It is possible 
that some of these participants did not teach in Louisiana between 2014/15 and 2016/17. Some might have 
entered teaching after 2016/17. The Louisiana Department of Education indicated that many of the records 
were not matched because the program records submitted by teacher preparation providers or school 
systems did not include Social Security number or other data necessary for matching. Name changes or data 
reporting errors could also account for the inability to match some of the program records. 


e Removed duplicate records from the 491 records with identifiers. For the 23 pairs of duplicate records that 
differed in cohort designation, the record with the earlier cohort was removed.’ This resulted in 468 unique 
records of Believe and Prepare participants in the analytic dataset. 


e Recoded district of residency identifiers. District identifiers for Recovery School Districts? in the Believe and 
Prepare data did not match the district identifiers in the employment dataset. For districts with the same 
name but with different district identifiers, the study team replaced the district identifier in the Believe and 
Prepare dataset (105 records) with the district identifier from the employment dataset. An additional 10 
records in the Believe and Prepare dataset for Recovery School Districts did not have a close district name 
match in the employment file and were given researcher-assigned district identifiers for each unique 
residency district name. 


With duplicates removed, the sample included 713 unique Believe and Prepare pilot participants, 468 of them 
with identifiers and 245 without identifiers. 


Certification data from the Teacher Certification Management System. The certification dataset consisted of 
134,794 records of teacher certifications issued from 2014/15 through 2017/18 for 26,360 unique teachers. 
Variables included unique random individual identification number, certification type code, certification number, 
valid to/from dates, issue date, program identifier, and program place name. Each row was a unique 
certification/endorsement area associated with a specific teaching certificate. Teachers with more than one 
certificate or more than one endorsement per certificate appeared in multiple rows in the data. The study team 
prepared the data using the following key steps: 


e Created a variable to indicate the first certification issued to each teacher between 2014/15 and 2017/18. 


e Created a variable to indicate whether teachers received a Level 1 professional certificate after 2014/15. 


1 One of the 23 pairs also differed in school of residency. 

? In 2003 the Louisiana legislature created the Recovery School District to turn around the lowest performing schools in the state. From 
November 2005 through June 2018 the Recovery School District directly operated dozens of schools and authorized charter schools 
(Louisiana Department of Education, n.d.). 
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e Generated a certification subject variable using a classification system provided by the Louisiana Department 
of Education to classify subjects. 


e Reshaped data to make each row a unique teacher. 


Teaching records from the Profile of Educational Personnel. The employment dataset consisted of 744,092 records 
of 69,863 unique teachers who taught in Louisiana’s public schools between 2014/15 and 2018/19. Variables 
included unique random individual identification number, school year, district identifier and name, school 
identifier and name, gender, race/ethnicity, years of experience, highest level of education, academic area, and 
course information (name, type, and description). Each record was a unique course that the teacher taught in a 
particular school year. To prepare the data for analysis, the study team performed the following key steps: 


e Created binary variables for race/ethnicity, gender, and highest level of education to indicate whether 
teachers were White, were female, or had at least a master’s degree. 


e Created three retention variables, for retention in state, retention in district, and retention in school. The 
study team identified the first year in which teachers appeared in the employment dataset. Then, the study 
team created binary variables to indicate whether teachers had records in the employment dataset one year 
after first appearing in the file for any district in Louisiana, for the same district, and for the same school. 
Teachers who taught in multiple districts or in multiple schools in the year they first appeared in the personnel 
records were considered to have been retained in the same district or school if any one of the employment 
district or school identifiers was the same the following year. 


e Created two binary variables to indicate whether teachers in the first year of employment were employed in 
the district in which they served their residency and whether they were employed in the same school as during 
their residency. Teachers who taught in multiple districts or schools in their first year of employment were 
considered to have taught at their district or school of residency if the residency district or school matched 
any of the multiple employment districts or schools. 


e Created a variable in the first record in the employment file for subject(s) taught based on the course 
description variables (for example, math, special education, trade, and industry) and academic area 
description variables (for example, elementary, middle grades, and vocational) using a classification system 
provided by the Louisiana Department of Education. 


e Created a unique record for each teacher. The study team aggregated records for individual teachers from the 
original dataset to create a unique record for each teacher with variables on teacher characteristics, first year 
of employment, and retention. 


Merger of teacher-level data. After preparing each of the three files provided by the Louisiana Department of 
Education, the study team merged the files using the common teacher identifier, retaining all unique records from 
each file. Binary variables were created to indicate whether a record in one file had a match in another. These 
merged data were used to identify the samples of early career Believe and Prepare participants for the various 
outcomes, as discussed later in this appendix. These data also identified 10,275 other early career teachers in 
Louisiana. 


School demographic data. Data on school characteristics were obtained for 2014/15-2017/18 from the Louisiana 
Department of Education’s Data Center webpage (https://www.louisianabelieves.com/resources/library/data- 
center) and from the National Center for Education Statistics Common Core of Data (National Center for Education 
Statistics, 2016, 2017, 2018, 2019). The Louisiana Department of Education’s Data Center provided school 
demographic data, including total enrollment, percentages of students by race/ethnicity, percentage of 
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economically disadvantaged students, and percentage of English learner students. The National Center for 
Education Statistics Common Core of Data provided data on administrative characteristics, such as school urban- 
centric locale codes, charter school designation, and instructional level (primary, middle, high, other).? The study 
team performed the following key steps in preparing the data: 


e Merged the school demographic and school administrative data files by school identifier and school year. 


e  Recoded variables for school locale and level. The study team used the four major school locale categories 
from the Common Core of Data (city, suburb, town, and rural) for descriptive analyses. For regression analyses 
the study team recoded school locale to a binary variable indicating whether the school was urban (city) or 
not (suburb, town, and rural). Similarly, the study team used the four Common Core of Data categories for 
school grade level (primary, middle, high, and other) for descriptive analyses and recoded school level for 
regression analyses to a binary variable indicating whether the school was a primary school (with a value of 
1) or anonprimary school (with a value of 0). (See box 1 in the main report for definition of school grade level.) 


District demographic data. Data on district characteristics were obtained for 2014/15—2017/18 from the Louisiana 
Department of Education’s Data Center webpage. Data included total enrollment, percentage of economically 
disadvantaged students, percentage of racial/ethnic minority students, and percentage of English learner students 
at the district level. 


Merger of teacher data with school and district data. The study team merged the teacher data (from the file after 
merging the three datasets provided by the Louisiana Department of Education) with the school and district 
characteristic datasets. Believe and Prepare pilot participants were linked to two sets of school characteristics: 
residency school/district characteristics using the residency school/district identifier and cohort year and 
employment school/district characteristics using school/district identifier and the first year it appeared in the 
employment dataset. Other teachers in the state who did not participate in the Believe and Prepare pilot were 
linked only to characteristics of the school and district of employment. For teachers who taught at multiple schools 
in their first year of teaching, the study team took the average across schools for each characteristic. 


Missing data 


The data files used for the study included teacher and school records with missing data. 


Teacher-level data. Of the 713 Believe and Prepare pilot participants, 71 records (10 percent) did not have 
identifiers for school of residency and therefore did not have associated demographic data for school of 
residency.’ Of the records for the 468 Believe and Prepare pilot participants with identifiers, 36 records (8 percent) 
did not have identifiers for school of residency. 


Of the 468 Believe and Prepare pilot participants with identifiers, 430 (92 percent) had at least one record in the 
employment dataset. Teacher demographic data were available only for these 430 teachers because only the 
employment dataset included demographic data. Of these 430 teachers, 11 (3 percent) were missing data on 
race/ethnicity, and none was missing data on gender or highest level of education. 


3 The available data included the percentage of students by proficiency level, a measure of school-level achievement for schools serving 
students in grades 3-8. Because a comparable measure of school achievement was not available for K—2 schools or secondary schools, 
school achievement was excluded from this study. 

4 The Louisiana Department of Education confirmed that unsuccessful matches to school identifiers, given the information reported to the 
department (for example, the name of the school submitted did not match the department’s records), was the primary reason these 
records did not have identifiers for school of residency. 
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Of the 10,275 early career teachers who did not participate in the Believe and Prepare pilot program, 1,173 (11 
percent) were missing data on race/ethnicity, 943 (9 percent) were missing data on gender, and 1,108 (11 percent) 
were missing data on highest level of education. 


Teachers who were missing data on a variable were excluded from relevant analyses. 


School and district demographic data. For schools and districts with missing data for a particular year, the study 
team used demographic data from the latest year available. Doing so resulted in no missing demographic data for 
school of residency among Believe and Prepare pilot participants who had nonmissing residency school identifiers, 
and no missing demographic data for schools of employment for the 430 Believe and Prepare pilot participants 
who had records in the employment file. Among the 10,275 other early career teachers in the state, 102 
(1 percent) were missing demographic data on school of employment. Records with missing demographic data 


were excluded from relevant analyses. 


Variables 


The data preparation process described above resulted in the set of teacher, school, and district variables used in 
the analysis and shown in table B1. 


Table B1. Summary of variables used in the analysis of participants in the Believe and Prepare pilot program, 


2014/15-2018/19 
ETE] (= 
Outcome 
Received Level 1 


certification after 2014/15 
Entered teaching 
Retention In the district 


Retention in the school 


Retention in the state 


Data source 


Derived from LDOE 
provided data 
Derived from LDOE 
provided data 
Derived from LDOE 
provided data 
Derived from LDOE 
provided data 


Derived from LDOE 
provided data 


Variable description 


1= Received Level 1 professional certificate after 2014/15 

0 = Did not receive Level 1 professional certificate after 2014/15 
1 = Entered teaching in Louisiana after 2014/15 

0 = Did not enter teaching in Louisiana after 2014/15 

1 = Retained in the district in the second year of teaching 

0 = Not retained in the district in the second year of teaching 

1 = Retained in the school in the second year of teaching 

0 = Not retained in the school in the second year of teaching 


1 = Retained in the state in the second year of teaching 
0 = Not retained in the state in the second year of teaching 


Believe and Prepare pilot program participation 


Believe and Prepare pilot 
cohort 


Residency school identifier 


Residency district identifier 


Teacher preparation 


provider name 


Teacher preparation subject 


First school of employment 
is residency school 
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LDOE: Believe and 
Prepare program 
records 

LDOE: Believe and 
Prepare program 
records 

LDOE: Believe and 
Prepare program 
records 

LDOE: Believe and 
Prepare program 
records 

LDOE: Believe and 
Prepare program 
records 

Derived from LDOE 
provided data 


1 = Cohort 1 (2014/15); 2 = Cohort 2 (2015/16); 3 = Cohort 3 
(2015/16 or 2016/17) 


Six-character alphanumeric code indicating the school in which the 
teacher served the residency 


Three-character alphanumeric code indicating the district in which 
the teacher served the residency 


Name of teacher preparation provider 


Subject while enrolled in the teacher preparation program 


1 = First school of employment was school of residency 
0 = First school of employment was not school of residency 


WETat-}<)(- Data source Variable description 
Teacher demographics 
Race/ethnicity LDOE: Employment American Indian or Alaska Native, Asian, Black or African American, 
dataset Hispanic, Native Hawaiian or Pacific Islander, White, multiple (two 
or more races/ethnicities) 
Dummy variable for White used in regression analysis (1 = White; 
0 = other race/ethnicity) 


Gender LDOE: Employment 1 =Female 
dataset 0 = Male 

Highest level of education LDOE: Employment Dummy variable for possession of master’s degree or higher used in 
dataset regression analysis based on data provided (1 = master’s degree or 


higher, including master’s degree, master’s degree +30 additional 
graduate units, educational specialist, and doctorate; 0 = less than a 
master’s degree, including less than high school, high school, 
trade/technical certificate, one year of college, two years of college, 
associate degree, three years of college, bachelor’s degree) 


Certification 
Certification type LDOE: Certification Type of certification (for example, Level 1 professional certificate, 
dataset Level 2 professional certificate, Level 3 professional certificate) 
First certificate LDOE: Certification The first certification issued between 2014/15 and 2017/18 
dataset 
Certification subject LDOE: Certification One of 17 subject categories in which teachers attained 
dataset and subject certification® 
designations provided by 
LDOE 
Certification issue year LDOE: Certification Certification issue date 
dataset 


Year first appear LDOE: Employment First year of appearance in the employment dataset, between 
dataset 2014/15 and 2018/19 
Employment school LDOE: Employment Code indicating the school in which teachers taught in their first 
identifier dataset year of teaching (that is, the school associated with the earliest 
record in the employment dataset) 
Employment district LDOE: Employment Code indicating the district in which teachers taught in their first 
identifier dataset year of teaching (that is, district associated with the earliest record 
in the employment dataset) 
Course category LDOE: Employment Category of the course taught in teachers’ first year of teaching 
eS dataset 
description 
Academic area description LDOE: Employment Academic area of the course taught in teachers’ first year of 
dataset teaching (all grades [K-12], elementary—middle [1-8], kindergarten 


[PK-K], elementary [1-5], middle school [6-8], secondary [9-12], 
non-public [K-12], ROTC, vocational [6—12]) 


Employment subject LDOE: Employment One of 22 subject categories teachers taught in a particular year? 
dataset 


Total enrollment LDOE Data Center Number of students enrolled at the school 


Percentage of racial/ethnic LDOE Data Center Percentage of racial/ethnic minority students at the school 
minority students 


Percentage of English LDOE Data Center Percentage of English learner students at the school 
learner students 
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ETE] (= Data source Variable description 


Percentage of economically | LDOE Data Center Percentage of economically disadvantaged students at the school 
disadvantaged students 
School locale NCES: Common Core of City, suburb, town, and rural 

Data Dummy variable for urban locale used in regression analysis 

(1 = city; 0 = suburb, town, or rural) 

School grade level NCES: Common Core of Primary, middle, high, other 

Data Dummy variable for primary used in regression analysis 

(1 = primary; 0 = other school grade level)] 

Charter NCES: Common Core of Charter or not charter school (1 = charter school; 0 = not charter 

Data school) 

istrict 

Total enrollment LDOE Data Center Number of students enrolled in the district 
Percentage of racial/ethnic LDOE Data Center Percentage of racial/ethnic minority students in the district 
minority students 
Percentage of English LDOE Data Center Percentage of English learner students in the district 
learner students 
Percentage of economically | LDOE Data Center Percentage of economically disadvantaged students in the district 
disadvantaged students 
Percentage of students LDOE Data Center Percentage of students scoring at least basic in English language 
scoring at least basic in arts (average across grades 3-8) in the district 


English language arts 
(average across grades 3-8) 


Percentage of students LDOE Data Center Percentage of students scoring at least basic in math (average 
scoring at least basic in across grades 3-8) in the district 

math (average across 

grades 3-8) 


LDOE is Louisiana Department of education. NCES is National Center for Education Statistics. 

a. The subject categories are art, early childhood, elementary education, English/language arts—middle grades, English/language arts—secondary grades, 
foreign languages, health and physical education, math—middle grades, math-secondary grades, music, science—middle grades, science—secondary grades, 
social studies—middle grades, social studies—secondary grades, special education, speech, and other. 

b. The subject categories are all grades English/language arts, math, science, and social studies, and speech; art; early childhood; elementary 
English/language arts; elementary; foreign languages; health and physical education; middle grades English/language arts; middle grades math; middle 
grades science; middle grades social studies; music; secondary English/language arts; secondary math; secondary science; secondary social studies; special 
education; and other. 

Source: Authors’ compilation based on data provided by the Louisiana Department of Education, data obtained from the department’s Data Center webpage 
(https://www.louisianabelieves.com/resources/library/data-center), and data obtained from National Center for Education Statistics (2016, 2017, 2018, 
2019). 


Sample 

The Believe and Prepare pilot program served primarily undergraduate teacher candidates. However, the 
Louisiana Department of Education indicated that the pilot program also included participants from university- 
based post-baccalaureate programs and alternate programs, as well as some certified or in-service teachers 
pursuing additional certifications. For this study the Louisiana Department of Education was interested primarily 
in early career Believe and Prepare pilot participants. Therefore, in consultation with the Louisiana Department 
of Education, the study team applied the following set of exclusion criteria to the sample of 468 Believe and 
Prepare pilot participants with identifiers to remove participants who were likely to be experienced teachers or 
whose experience could not be determined using the data available: 


e Based on certification records, teachers who were issued a Level 1 professional certificate in 2014/15, that is, 
prior to or during the first year of the Believe and Prepare pilot program (n = 4). 
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e Based on certification records, teachers whose first certificate was issued between 2014/15 and 2017/18 and 
required at least three years of experience, such as a Level 2 or Level 3 professional certificate> or a Type A or 
Type B certificate (n = 32). 


e Based on certification records, teachers who were not issued a certificate between 2014/15 and 2017/18 but 
who were teaching with a Level 1 professional certificate or higher (Levels 2 and 3 professional certificates, 
Types A or B certificates) when they first appeared in the employment records, which indicates that they had 
attained a Level 1 professional certificate or higher prior to 2014/15 (n = 58). 


e Based on certification records, teachers whose first certification issued between 2014/15 and 2017/18 was a 
certification type for which level of experience was indeterminate, including career, technical, trade, and 
industrial education certificates, out-of-state certificates, or out-of-field certificates (n = 17). 


e Based on certification records, teachers who were not issued a certification between 2014/15 and 2017/18 
but whose earliest known certification reported in the employment records was one of the certification types 
for which the level of experience is indeterminate (n = 7). 


e Teachers who did not have a matching record in the certification or the employment records for all years 
provided to the research team. These teachers had identifiers in the study data, which meant that they were 
matched in the Louisiana Department of Education administrative records and thus likely had attained 
certification or taught outside of the date range available and were likely to be experienced teachers (n = 14). 


The application of these exclusion criteria resulted in a sample of 336 early career Believe and Prepare pilot 
participants. 


Where applicable, the study team applied additional criteria to construct the analytic samples for each research 
question (see figure B1 for a visual description of the samples): 


e Research question 1. The sample included all 713 Believe and Prepare pilot participants. As described in the 
data preparation section, data on participant cohort, teacher preparation provider, teacher preparation 
subject, school, and district of residency were available for these 713 teachers. 


e Research question 2. The sample for research question 2 included 268 participants who attained a Level 1 
professional certificate. The sample for research question 2a included 336 participants (268 who attained a 
Level 1 professional certificate and 68 who did not). 


e Research question 3. For research questions 3 and 3a, because the outcome of interest was entering teaching, 
the study team excluded 58 teachers who were already teaching in 2014/15.° The sample for research 
question 3 included 255 participants who entered teaching in Louisiana between 2015/16 and 2018/19. The 


5 Teachers can apply for a Level 2 professional certificate after teaching for a minimum of three years, and they can apply for a Level 3 
professional certificate after teaching for five years and meeting other licensure requirements. 

6 The study team included these 58 participants in the sample of 336 participants because their certification status indicated that they were 
early career participants. That is, these participants were teaching with a practitioner license or did not receive their Level 1 professional 
certificate until after 2014/15. However, the study team excluded these teachers from the sample for research questions 3 and 3a because 
this question specifically addresses the outcome of entering employment after 2014/15. These 58 teachers were already teaching in 
2014/15, and the study team could not determine whether they first taught in 2014/15 or in a prior year. Among the 58 participants, 44 
were issued a practitioner license between 2014/15 and 2017/18, 6 were issued a Level 1 professional certificate after 2014/15, 6 were not 
issued any certification between 2014/15 and 2017/18 and had no certification in their earliest record in the employment data, and 2 were 
not issued any certification between 2014/15 and 2017/18. One participant held an ancillary certificate, and the other held a practitioner 
license in the earliest record of teaching. 
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sample for research question 3a included 278 participants (255 participants who entered teaching and 23 who 
did not). 


e Research questions 4 and 5. The sample for analyses of retention included 245 early career Believe and 
Prepare teachers who gained employment in Louisiana public schools between 2015/16 and 2017/18. The 
study team arrived at the sample of 245 teachers by excluding the following participants from the pool of 336 
early career Believe and Prepare pilot participants: 


©. Participants who were already teaching in Louisiana public schools in 2014/15 (n = 58). 


© Participants who were not in the employment file between 2014/15 and 2018/19 (n = 23). Of these 23 
participants, 5 were issued a practitioner license, and 18 were issued a Level 1 professional certificate 
between 2014/15 and 2017/18. 


©. Participants whose first record in the employment file was for 2018/19 (n = 10). Because the employment 
file was limited to teaching records between 2014/15 and 2018/19, the study team did not have teaching 
records for 2019/20 to determine whether these 10 teachers returned to teaching in Louisiana public 
schools in the second year. 


The sample for research question 4b also included 10,275 other early career teachers in the state who did not 
participate in the Believe and Prepare pilot program. To identify the group of other early career teachers, the 
study team applied the same criteria used to identify the sample of early career Believe and Prepare teachers. 


Figure B1. Creation of analytic samples for certification, employment, and retention, by research question 


245 did not 
have a unique 
RQ1 identifier re 
sean bas determined to be 
Si experienced or 
seis based had inconclusive Certification Employment Retention 
on records 5 RQ3 RQs 4 and5 
provided to LDOE erpsnelee ne 
468 hada 268 received the 58 were already 
unique Level 1 professional teaching i 10 
identifier certificate apa Mine i 
Main analytic Main analytic 2014/15 employed in 
sample sample 2018/19 
336 were 336 ait 255 were 
determined to be cing tobe employed after 
early career ony cae 2014/15 
teachers teachers 
245 were 
68 did not : employed 
23 did not have a 
receive the Level record in the pease 5/16 
1 professional employment file 
certificate between 2014/15 
Excluded from sample and 2018/19 


LDOE is Louisiana Department of Education. RQ is research question. 
Source: Authors’ compilation. 


Data analysis 

Research question 1: What types of teacher preparation programs did Believe and Prepare pilot participants 
attend? How did their residency districts and schools compare with other districts and schools in the state? The 
study team first calculated the number of distinct teacher preparation programs, districts of residency, and 
schools of residency. To examine the distribution of participants across programs, districts, and schools, the study 
team also calculated the number of Believe and Prepare pilot participants at each teacher preparation program 
and at each district and school of residency. The study team compared the characteristics of the districts and 
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schools that participated in the Believe and Prepare pilot with all districts and schools in the state. Group 
differences that were 5 percentage points or greater were considered meaningful. 


Research question 2: In what subjects did early career Believe and Prepare pilot participants attain Level 1 
professional certificates between 2015/16 and 2017/18? The percentage of participants who attained a Level 1 
professional certificate in a particular subject was calculated as the number of participants who attained a Level 
1 professional certificate in that subject divided by the number of teachers who attained a Level 1 professional 
certificate in any subject between 2015/16 and 2017/18 (n = 268). Because teachers could attain a certification in 
multiple subjects, percentages do not sum to 100 percent. The percentage of participants who attained at least 
one Level 1 professional certificate in a high-need subject was calculated as the number of participants with a 
Level 1 professional certificate in at least one high-need subject divided by the number of participants who 
attained a Level 1 professional certificate in any subject between 2015/16 and 2017/18. High-need subjects 
included middle grades math or science, secondary math or science, and special education, as defined by the 
Louisiana Department of Education. 


Research question 2a: Was attainment of a Level 1 professional certificate associated with characteristics of the 
schools (enrollment, locale, type, charter school designation, and student demographics) in which participants 
taught during their residency? To identify the characteristics of pilot program participants and their school of 
residency that were associated with certification, the study team conducted regression analysis using multilevel 
logistic regression models with the following structure: 


(B1) Logit (Yijx) _ B, Cohort; + B2Xijx + B3Wyx + Up + Eijk 
where 


e Yi is the probability of attaining a Level 1 professional certificate for teacher i in school j in district k. 
Individuals were assigned a 1 if they received a Level 1 professional certificate after 2014/15 and a 0 if they 
did not achieve that outcome. 


e Xj is a vector of teacher covariates. Because teacher characteristics (race/ethnicity, gender, highest level of 
education) were available only in the employment dataset, these characteristics were not included in models 
for certification and employment. 


@ Wy. is a vector of school characteristics, including percentage of racial/ethnic minority students, percentage 
of English learner students, percentage of economically disadvantaged students, log of total enrollment, and 
binary variables indicating whether the school was a primary school, a charter school, or in an urban area. 
Characteristics of residency schools were used for analyses of certification and employment. 


e ux isa district random effect of residency districts, and €;;, is the individual error term. The study team used 
random instead of fixed district effects because the goal was to account for teacher clustering within districts 
(to obtain accurate standard errors), not to estimate the effect of each individual district. 


For each model the study team estimated separate intercepts for each cohort to account for unmeasured 
differences in program implementation and timing of the cohorts.’ The study team employed a backward variable 
selection approach that removed less significant covariates until the best possible (minimal) value of Akaike 


7 In considering whether to include both school and district characteristics, the study team observed a strong correlation between school 
and district characteristics, which could be explained by the small size and homogeneity of many districts. For a similar reason school and 
district random effects could not be identified simultaneously. The study team determined that the model with district-only random effects 
was adequate. Therefore, the models as reported included only teacher- and school-level fixed effects and only district-level random 
effects, as shown in model B1. 
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information criterion was achieved (Venables & Ripley, 2002). The goal was to find the best tradeoff between 
model complexity (the number of covariates) and the goodness-of-fit (explained variance). The selection process 
was implemented using a modification of R function “step()” adapted for generalized linear mixed models. 


Research question 3: In what subjects did early career Believe and Prepare pilot participants teach in their first 
year of employment in Louisiana public schools between 2015/16 and 2018/19? The percentage of early career 
Believe and Prepare pilot participants who taught courses in a particular subject was calculated as the number of 
participants who entered teaching in Louisiana public schools between 2015/16 and 2018/19 and who taught a 
course in that subject divided by the total number of participants who entered teaching in Louisiana public schools 
between 2015/16 and 2018/19 (n = 255). Because participants could teach courses in multiple subjects, 
percentages do not sum to 100 percent. The percentage of participants who taught a course in at least one high- 
need subject was calculated as the number of participants who taught at least one course in a high-need subject 
divided by the total number of participants who entered teaching in Louisiana public schools between 2015/16 
and 2018/19 (n = 255). 


Research question 3a: Was entry into teaching associated with characteristics of the schools (enrollment, locale, 
type, charter school designation, and student demographics) in which participants taught during their residency? 
This research question was addressed using models similar to those for research question 2a. The outcome is the 
probability of entering teaching in Louisiana after 2014/15. In addition to characteristics of schools of residency, 
a binary variable was included to indicate whether the teacher received a Level 1 professional certificate after 
2014/15. 


Research question 4: What percentages of early career Believe and Prepare teachers overall and by subject were 
retained in teaching positions in the state, in the district, or in the school for a second year between 2016/17 and 
2018/19? The study team calculated within-state, within-district, and within-school retention rates for early 
career Believe and Prepare pilot participants who entered teaching in Louisiana between 2015/16 and 2017/18 
as the ratios of the following two quantities: 


e Number of early career Believe and Prepare pilot participants who returned to teach in the state, district, or 
school in the second year of employment. 


e Number of early career Believe and Prepare pilot participants who entered teaching in Louisiana between 
2015/16 and 2017/18 (n = 245). 


The study team calculated the distribution of first-year retention rates (in-state, in-district, and in-school) of early 
career Believe and Prepare participants who entered teaching in Louisiana between 2015/16 and 2017/18 by 
subjects taught in the first year of employment. Special attention was given to high-need subjects. The retention 
rate for each subject was calculated as the number of participants who taught that subject in their first year of 
teaching and returned to teach in the state, district, or school in the second year of employment (regardless of 
the subject taught in the second year) divided by the number of participants who taught that subject. 


Research question 4a: Was retention associated with characteristics of teachers (gender, race/ethnicity, highest 
level of education, and whether they attained initial certification) and characteristics of schools (enrollment, locale, 
type, charter school designation, and student demographics) in which they were first employed? To identify the 
characteristics of early career Believe and Prepare pilot participants and their schools that were associated with 
retention (in state, in district, or in school), the study team conducted regression analysis using model B1 above, 
with the following modifications: 


e Yi is the probability of achieving an outcome of interest (retention in state, district, or school) for teacher jin 
school j in district k. For each outcome individuals were assigned a value of 1 if they returned to teaching in 
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the state, the same district, or the same school in the second year of employment and a value of 0 if they 
were in the sample for that outcome but did not return to teaching in the state, the same district, or the same 
school in the second year of employment. 


e Xi isa vector of teacher covariates, including five binary variables to indicate whether the teacher was White, 
was female, had at least a master’s degree, attained an initial certification after 2014/15, and was employed 
in the residency school in the first year of teaching. 


e Wy is a vector of characteristics of the teacher’s first school of employment, including log of total enrollment, 
percentage of economically disadvantaged students, percentage of racial/ethnic minority students, 
percentage of English learner students, and binary variables indicating whether the school was a primary 
school, a charter school, or in an urban area. 


© Ux isa district random effect of employment districts, and €;;, is the individual error term. 


The study team applied the same approach of estimating separate intercepts for each cohort and employed the 
same variable selection process as described above for the analyses predicting certification and employment. 


Research question 4b: How did the retention rates of early career Believe and Prepare teachers, teacher 
characteristics, and characteristics of the schools and districts in which they taught compare with those of other 
early career teachers in Louisiana? The study team compared early career Believe and Prepare pilot participants 
and other early career teachers in the state on retention rates and on several characteristics. For each retention 
outcome (in state, in district, or in school), the study team applied a chi-square test to determine whether a 
relationship existed between the outcome (retained or not) and preparation of early career teachers through the 
Believe and Prepare pilot program. For each teacher characteristic the study team also applied a chi-square test 
to determine whether the proportions of teachers across the different categories (for example, proportions of 
male teachers and female teachers) were similar between the group of early career teachers prepared through 
the Believe and Prepare pilot and the group of other early career teachers in Louisiana. The study team applied 
the two-sample t-test to assess the similarities across the two groups in total enrollment, percentages of 
economically disadvantaged students, percentages of racial/ethnic minority students, and percentages of English 
learner students in the district and school. 


Research question 5: What percentages of early career Believe and Prepare teachers entered teaching in the 
districts and in the schools in which they served their residency? The study team calculated the percentages of 
early career Believe and Prepare teachers who entered teaching in the schools and districts in which they served 
their residency as ratios of the following two quantities: 


e Number of teachers who were employed in the same school (or same district) as their residency school (or 
district). 


e Number of early career Believe and Prepare pilot participants who entered teaching in Louisiana between 
2015/16 and 2017/18 (n = 245). 


Research question 5a: Did retention rates differ between participants who entered teaching at their school of 
residency and participants who entered teaching at another school? The study team conducted regression 
analyses using model B1 from research question 2a with an additional binary term indicating whether the teacher 
was employed in the same school in which residency was served. The sign of the regression coefficient (beta) for 
the binary term indicates whether being employed in the school of residency was positively or negatively 
associated with within-school retention (see model 1 in table C6 in appendix C). The standard error indicates 
whether the relationship was statistically significant. 
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Use of regression results 


For research questions 2a, 3a, 4a and 5a the primary interest was to identify the sign and significance of various 
predictors. The regression results produced estimates of regression coefficients (betas) and standard errors that 
identified statistically significant predictors of outcomes. To facilitate interpretability, results for the main sample 
using the optimal models were reported as average marginal effects (see table C3 in appendix C). Average marginal 
effects, which were generated using the margins function of R’s margins package (Leeper, 2018), can be 
interpreted as the increase in the probability of the outcome (in percentage points) given a one-unit increase in 
the predictor after other teacher and school characteristics were accounted for. For binary variables, such as 
charter or noncharter, the average marginal effect is the difference in the probability of the outcome for teachers 
teaching at charter schools compared with teachers teaching at noncharter schools, after other factors were 
adjusted for. 


Sensitivity analysis 

The study team conducted one sensitivity analysis to assess the stability of the main results for certification, 
employment, and retention outcomes by varying how the cohort variable was handled in the analysis. The main 
models estimated cohort to account for unmeasured differences in program implementation and timing of the 
cohorts. However, cohort 1 was a near-perfect predictor of the outcomes (that is, most of cohort 1 achieved the 
outcome). In the sensitivity analysis the study team limited the samples to cohorts 2 and 3 to assess whether and 
how regression coefficients were affected after cohort 1 was removed. The sensitivity analysis showed that the 
results were robust against the changes in model specifications and variations in the sample. All the variables that 
were significant in the main models had the same direction of association (the sign of regression coefficients) with 
the outcomes across all models. The strength of the association was comparable (same order of magnitude of 
coefficients) in all cases. 


Supplemental analysis 


The study team conducted supplemental analyses for research questions 2a (Was attainment of a Level 1 
professional certificate associated with characteristics of the schools in which participants taught during their 
residency?) and 3a (Was entry into teaching associated with characteristics of the schools in which participants 
taught during their residency?) using an alternative sample that included the 245 Believe and Prepare pilot 
participants who were not matched to the administrative records of the Louisiana Department of Education (that 
is, participants without identifiers). These analyses were considered supplemental because of the uncertainty 
about why the Louisiana Department of Education was unable to match a fairly large percentage (34 percent) of 
Believe and Prepare participants with records in the Department’s Profile of Education Personnel data. The 
supplemental analysis assumed that none of the 245 participants attained certification or entered teaching in 
Louisiana and that they were all early in their career. Their certification and employment outcomes were coded 
as O when they were included in the analysis. This approach resulted in a sample of 581 participants for research 
question 2a and a sample of 523 participants for research question 3a (58 participants who were already teaching 
in 2014/15 were excluded). The study team examined whether certification (research question 2a) and entering 
teaching (research question 3a) were associated with characteristics of schools of residency using the same 
models as those used for the main analyses. The results of the supplemental analyses are presented in appendix D. 
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Appendix C. Supporting analysis 


This appendix provides additional findings from the analyses, including comparisons of the composition of the 
districts and schools of residency of Believe and Prepare pilot participants and all districts and schools in the state, 
detailed regression results and sensitivity analyses for all outcomes, and comparisons of characteristics of Believe 
and Prepare pilot participants with identifiers and participants without identifiers. 


Findings for all Believe and Prepare pilot participants 


This section presents additional findings on the distribution of all 713 Believe and Prepare pilot participants 
between 2014/15 and 2016/17 by teacher preparation program and district and school of residency, as well as by 
the characteristics of their districts and schools of residency. 


Believe and Prepare pilot participants were enrolled in 27 teacher preparation programs (figure C1) and taught in 
45 districts (figure C2) and 217 schools during residency. Compared with all districts in the state, Believe and 
Prepare pilot participants’ districts of residency had a higher average percentage of economically disadvantaged 
students (table C1). Compared with all schools in the state, Believe and Prepare pilot participants’ schools of 
residency had a higher average percentage of racial/ethnic minority students (table C2) and consisted of a higher 
percentage of city schools and charter schools (figure C3). 


Figure C1. Believe and Prepare pilot participants were enrolled in 27 teacher preparation programs, with 
nearly half of participants enrolled in the top 5 programs, 2014/15-2016/17 
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Note: Among the 713 pilot participants in the Believe and Prepare program records, the Louisiana Department of Education could not match 245 participants 
(34 percent) to its employment records. Those participants were therefore not assigned an identifier and were not included in the analysis of certification 
or employment outcomes. 

Source: Authors’ analysis of data from the Believe and Prepare Pilot Participants dataset provided by the Louisiana Department of Education. 
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Figure C2. Believe and Prepare pilot participants served a residency in 45 school districts, with 40 percent of 
participants completing their residency in the top 5 districts, 2014/15—-2016/17 


Orleans Parish 

East Baton Rouge Parish 

Lincoln Parish 

Lafayette Parish 

St. Bernard Parish 

Rapides Parish 

Lafourche Parish 

RSD - Knowledge is Power Program (KIPP) N.O. 
St. Landry Parish 

Ascension Parish 

Terrebonne Parish 

RSD - New Beginnings Schools Foundation 
Calcasieu Parish 

Ouachita Parish 

RSD - Collegiate Academies 

St. Charles Parish 

Caddo Parish 

Lusher Charter School 

RSD - New Orleans College Preparatory Academies 
Bossier Parish 

RSD - Arise Academy 

St. Martin Parish 

Pointe Coupee Parish 

Union Parish 

Evangeline Parish 

RSD - Choice Foundation 

Other? 


Note: n= 703. 


7 (1%) 
0 10 20 30 40 50 60 


Number of participants 


a. The 19 residency districts that had five or fewer participants. 
Source: Authors’ analysis of data provided by the Louisiana Department of Education. 


Table C1. The student composition characteristics of Believe and Prepare pilot districts of residency and of all 


districts in the state, 2014/15-2016/17 


Believe and Prepare pilot 


districts of residency All districts in the state 


(ie) (n= 81) 
Siclalerlae| Syelalerlae) 
District characteristic deviation Median deviation Median 
Total enrollment 9,914 10,891 5,221 8,922 11,511 4,337 
Percentage of economically disadvantaged students 74.45 13.79 73.72 69.31 15.68 71.46 
Percentage of racial/ethnic minority students 54.81 23.67 51.68 54.44 22.73 52.67 
Percentage of English learner students 2.43 2.50 1.57 1.59 2.22 0.86 


Note: State means were calculated using 2017/18 data. 


Source: Authors’ analysis of data provided by the Louisiana Department of Education and data obtained from the department’s Data Center webpage 
(https://www.louisianabelieves.com/resources/library/data-center ) 


Table C2. The student composition characteristics of Believe and Prepare pilot schools of residency and of all 
schools in the state, 2014/15-2016/17 


Believe and Prepare pilot 


Yel avete) 3 fey residency All schools Ta) id ats) state 
(n = 217) (n = 1,409) 
Siclarerlae! Siclalerlae! 

Yel aToto) Mol a¥-Ta-Leiksl asia (a Mean deviation Median Mean deviation Median 
Total enrollment 643 385 546 529 328 456 
Percentage of economically disadvantaged students 75.49 17.02 78.57 71.97 18.16 75.10 
Percentage of racial/ethnic minority students 66.64 27.99 69.72 59.47 30.95 59.27 
Percentage of English learner students 3.02 4.90 1.17 3.23 6.22 0.98 


Note: State means were calculated using 2017/18 data. 
Source: Authors’ analysis of data provided by the Louisiana Department of Education and data obtained from the department’s Data Center webpage 
(https://www.louisianabelieves.com/resources/library/data-center). 


Figure C3. Other characteristics of Believe and Prepare pilot schools of residency differed, on average, from 
that of all schools in the state in grade level, locale, and charter school designation status, 2014/15-2016/17 
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Note: Percentages might not sum to 100 because of rounding. 
Source: Authors’ analysis of data provided by the Louisiana Department of Education and data obtained from National Center for Education Statistics (2016, 
2017, 2018, 2019). 


Findings for early career Believe and Prepare pilot participants with certification or employment 
records 

This section presents additional findings related to certification, employment, and retention for the 336 
participants who met the study criteria for early career teachers or teacher candidates during the Believe and 
Prepare pilot, the primary group of interest to the Louisiana Department of Education. 


Findings from regressions on certification, employment, and retention for early career Believe and Prepare pilot 
participants. The study found statistically significant predicators for four of the five outcomes examined (tables 
C3-C6). 


Table C3. Predictors of certification and of retention outcomes in the school, district, or state for early career 
Believe and Prepare pilot participants, 2015/16-2018/19 


Average 
Outcome and predictor Estimate marginal effect Slilalellacmlanele 
Certification (characteristics of residency school) 
Residency school—charter —1.54** —24.20 0.48 
Residency school—primary 0.89* 11.06 0.37 
Retention in school (characteristics of employment school) 
Percentage of economically disadvantaged students 0.05** 0.81 0.02 
Percentage of racial/ethnic minority students —0.05*** —0.90 0.01 
Retention in district (characteristics of employment school) 
Percentage of economically disadvantaged students 0.05** 0.79 0.02 
Percentage of racial/ethnic minority students —0.05*** -0.72 0.01 
Is a primary school —0.73* —11.21 0.37 
Retention in state (characteristic of employment school) 
Received a Level 1 professional certificate 1.11* 12.58 0.47 


* Significant at p < .05; ** significant at p < .01; *** significant at p < .001. 

Note: The table shows results only for predictors that are significant. There were no significant predictors for employment. Estimates are unstandardized 
coefficients. 

Source: Authors’ analysis of data provided by the Louisiana Department of Education and publicly available data from the department’s Data Center webpage 
(https://www. louisianabelieves.com/resources/library/data-center) and from National Center for Education Statistics (2016, 2017, 2018, 2019). 
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Table C4. Regression results for predictors of attainment of Level 1 professional certificates for early career 
Believe and Prepare pilot participants, 2014/15-2017/18 


Predictor Model 1 WY KeYe (=) ie.) 
Cohort 1 0.02 —-0.75 
(3.45) (2.80) 
Cohort 2 0.03 -0.75 
(3.23) (2.51) 
Cohort 3 —0.36 —1.14 
(3.23) (2.52) 
Log (total enrollment in residency school) 0.48 0.54 
(0.39) (0.35) 
Percentage of economically disadvantaged students in residency school -0.01 na 
(0.03) 
Percentage of racial/ethnic minority students in residency school —0.01 -0.01 
(0.02) (0.01) 
Percentage of English learner students in residency school 0.00 na 
(0.05) 
Residency school is a primary school 0.94* 0.89* 
(0.40) (0.37) 
Residency school is in an urban area -0.05 na 
(0.50) 
Residency school is a charter school —1.58** —1.54** 
(0.59) (0.49) 
Akaike information criterion 264.20 258.38 
Bayesian information criterion 305.34 288.29 
Log likelihood —121.10 -121.19 
Number of observations 311 311 
Number of groups: Residency district identifier 36 36 
Variable: Residency district identifier (intercept) 0.22 0.23 


* Significant at p < .05; ** significant at p < .01. 

na indicates that the variable was removed during the variable selection process. 

Note: Estimates are unstandardized coefficients. Values in parentheses are standard errors. Model 1 includes all available covariates. Model 1a is the 
resulting model after applying the variable selection approach that removed less significant covariates until the best possible (minimal) value of Akaike 
information criterion was achieved. 

Source: Authors’ analysis of data provided by the Louisiana Department of Education and publicly available data from the department’s Data Center webpage 
(https://www.louisianabelieves.com/resources/library/data-center ) and from National Center for Education Statistics (2016, 2017, 2018, 2019). 
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Table C5. Regression results for predictors of entering teaching in Louisiana for early career Believe and 


Prepare pilot participants, 2014/15—2018/19 


dactellanelg Model 1 Model 1a 
Cohort 1 13.78 19.27 
(1,414.00) (1,436.07) 

Cohort 2 —2.46 2.75*** 
(5.14) (0.44) 

Cohort 3 —2.18 3: 7"** 
(5.16) (0.54) 

Teacher received Level 1 professional certificate -1.54 na 
(1.12) 

Log (total enrollment) 0.94 na 
(0.67) 

Percentage of economically disadvantaged students in residency school 0.02 na 
(0.04) 

Percentage of racial/ethnic minority students in residency school -0.01 na 
(0.03) 

Percentage of English learner students in residency school —0.08t —0.07+ 
(0.04) (0.04) 

Residency school is a primary school —0.12 na 
(0.68) 

Residency school is in an urban area -0.11 na 
(0.89) 

Residency school is a charter school -0.42 na 
(1.09) 

Akaike information criterion 162.33 153.38 

Bayesian information criterion 205.47 171.45 

Log likelihood —69.17 —71.69 

Number of observations 269 274 

Number of groups: Residency district identifier 36 37 

Variable: Residency district identifier (intercept) 1.06 1.58 


t Significant at p < .10; *** significant at p < .001. 
na indicates that the variable was removed during the variable selection process. 


Note: Estimates are unstandardized coefficients. Values in parentheses are standard errors. Model 1 includes all available covariates. Model 1a is the 
resulting model after applying the variable selection approach that removed less significant covariates until the best possible (minimal) value of Akaike 


information criterion was achieved. 


Source: Authors’ analysis of data provided by the Louisiana Department of Education and publicly available data from the department’s Data Center webpage 
(https://www.louisianabelieves.com/resources/library/data-center) and from National Center for Education Statistics (2016, 2017, 2018, 2019). 


REL 2021-079 


Table C6. Regression results for predictors of retention in the school, district, or state in the second year of 
teaching for early career Believe and Prepare teachers, 2015/16-2018/19 


iXcinclaia(elaMlamcvelarele)| Retention in district Retention in state 
actellanelg Model 1 Model 1a Model 1 Model 1a Model 1 Model 1a 
Cohort 1 -1.35 1.42 —1.76 0.18 21.62 0.96 
(2.85) (1.15) (3.33) (1.25) (2,301.23) (1.19) 
Cohort 2 -1.14 1.44 -1.16 0.99 6.08 1.54** 
(2.69) (0.87) (3.16) (0.98) (3.97) (0.51) 
Cohort 3 —1.17 1.36 -1.56 0.74 6.40 1.88** 
(2.70) (0.89) (3.15) (1.00) (3.97) (0.58) 
Teacher received Level 1 professional 0.30 na 0.60 0.66 1.31* 1.11* 
certificate 
(0.44) (0.47) (0.43) (0.54) (0.47) 
Teacher is White —0.69 —0.55 —0.49 -0.49 0.46 na 
(0.42) (0.38) (0.45) (0.42) (0.58) 
Teacher is female 0.17 na 0.42 na 0.43 na 
(0.46) (0.50) (0.59) 
Teacher has higher than a bachelor’s degree —0.22 na 0.10 na 0.48 na 
(0.47) (0.52) (0.70) 
Same residency and employment school 0.47 0.41 0.04 na —0.35 na 
(0.35) (0.34) (0.39) (0.50) 
Log (total enrollment in employment school) 0.33 na 0.24 na 0.53 na 
(0.34) (0.41) (0.49) 
Percentage of economically disadvantaged 0.04* 0.05** 0.06** 0.05** -0.01 na 
students in employment school 
(0.02) (0.02) (0.02) (0.02) (0.03) 
Percentage of racial/ethnic minority —0.04** —0.05*** —0.05** —0.05*** -0.01 na 
students in employment school 
(0.01) (0.01) (0.02) (0.01) (0.02) 
Percentage of English learner student in -0.01 na 0.01 na 0.04 na 
employment school 
(0.04) (0.05) (0.06) 
Employment school is a primary school —0.45 —0.44 —0.87* —0.73* —0.56 na 
(0.35) (0.33) (0.41) (0.37) (0.52) 
Employment school is in an urban area —0.39 na 0.63 na -0.51 —0.76t 
(0.45) (0.56) (0.69) (0.46) 
Employment school is a charter school —0.02 na 0.96 na —0.09 na 
(0.52) (0.67) (0.70) 
Akaike information criterion 276.13 265.28 252.73 245.27 171.89 164.40 
Bayesian information criterion 331.28 296.30 307.88 276.41 227.04 185.29 
Log likelihood —122.07 —123.64 —110.37 —113.64 -69.95 -76.20 
Number of observations 232 232 232 235 232 240 
Number of groups: Employment residency 49 49 49 51 49 51 
identifier 
Variable: Employment residency identifier 0.01 0.00 0.36 0.21 0.00 0.00 


(intercept) 
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t Significant at p < .10; * significant at p < .05; ** significant at p < .01; *** significant at p < .001. 

na indicates that the variable was removed during the variable selection process. 

Note: Estimates are unstandardized coefficients. The values in parentheses are standard errors. Model 1 includes all available covariates. Model 1a is the 
resulting model after applying the variable selection approach that removed less significant covariates until the best possible (minimal) value of Akaike 
information criterion was achieved. 

Source: Authors’ analysis of data provided by the Louisiana Department of Education and publicly available data from the department’s Data Center webpage 
(https://www.louisianabelieves.com/resources/library/data-center ) and from National Center for Education Statistics (2016, 2017, 2018, 2019). 


Comparison of retention rates for early career Believe and Prepare teachers and other early career teachers in the 
state. Among the samples of early career Believe and Prepare teachers and other early career teachers, teachers 
who attained a Level 1 professional certificate had higher retention rates than those who did not (figures C4 and 
C5).° The study team conducted additional analysis comparing the subsample of teachers who attained a Level 1 
professional certificate after 2014/15 and the subsample of teachers who did not. The differences in retention 
rates between early career Believe and Prepare teachers and other early career teachers for these two subsamples 
were not statistically significant (figures C6 and C7). 


Figure C4. Among early career Believe and Prepare teachers, those who attained a Level 1 professional 
certificate between 2015/16 and 2017/18 had significantly higher within-state and within-district retention 
rates than those who did not 


Percent of teachers 
@ With a Level 1 certificate (n = 197) @ Without a Level 1 certificate (n = 48) 
100 92 
79 
80 75 74 
62 60 

60 
40 
20 

0 

Retention in the state*** Retention in the district* Retention in the school 


* Significant at p < .05; *** significant at p < .001. 
Source: Authors’ analysis of data provided by the Louisiana Department of Education. 


° Forty-eight early career Believe and Prepare teachers did not attain a Level 1 professional certificate between 2015/16 and 2017/18. 
Among them, 22 teachers (46 percent) attained a practitioner license, and 26 teachers (54 percent) did not attain any certificate between 
2014/15 and 2017/18. Of those 26 teachers, 23 (88 percent) did not have a certificate associated with their teaching record between 
2014/15 and 2018/19, and 3 (12 percent) had an ancillary certificate or a temporary-authority-to-teach associated with their earliest 
teaching records. Among the 6,304 other early career teachers in the state who did not attain a Level 1 professional certificate between 
2014/15 and 2017/18, 1,345 (21 percent) attained a practitioner license, and 4,959 (79 percent) were not issued any certificate between 
2014/15 and 2017/18. Of those 4,959 teachers, 3,626 (73 percent) teachers had no certification associated with any of their teaching 
records between 2014/15 and 2018/19. The remaining 1,333 (27 percent) teachers had a certificate associated with their teaching record 
in one or more years (that is, they attained the certificate prior to 2014/15 or after 2017/18 ). Among the 1,333 teachers, 667 (50 percent) 
were teaching with a temporary-authority-to-teach certificate, 499 (37 percent) were teaching with an ancillary certificate, 106 (8 percent) 
were teaching with a practitioner license, and 61 (5 percent) were teaching with a professional certificate attained after 2017/18. 
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Figure C5. Among other early career teachers in the state, those who attained a Level 1 professional 
certificate between 2015/16 and 2017/18 had significantly higher within-state, within-district, and within- 
school retention rates than those who did not 


Percent of teachers 
With a Level 1 certificate (n = 3,971) B Without a Level 1 certificate (n = 6,304) 
100 
90 
80 
64 
60 56 
48 
40 
20 
0 
Retention in the state*** Retention in the district*** Retention in the school*** 


*** Significant at p < .001 
Source: Authors’ analysis of data provided by the Louisiana Department of Education. 


Figure C6. Among all teachers who attained a Level 1 professional certificate in the state between 2015/16 
and 2017/18, retention rates into the second year of teaching did not differ significantly between early career 
Believe and Prepare teachers and other early career teachers 


Percent of teachers 
m Believe and Prepare early-career teachers (n = 197) w Other early-career teachers (n = 3,971) 
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60 
40 
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Source: Authors’ analysis of data provided by the Louisiana Department of Education. 
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Figure C7. Among all teachers who did not attain a Level 1 professional certificate in the state between 
2015/16 and 2017/18, retention rates into the second year of teaching did not differ significantly between 
early career Believe and Prepare teachers and other early career teachers 


Percent of teachers 
m Early career Believe and Prepare teachers (n = 48) @ Other early career teachers (n = 6,304) 
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Retention in the state Retention in the district Retention in the school 


Source: Authors’ analysis of data provided by the Louisiana Department of Education. 


Comparison of teacher characteristics and characteristics of employment districts and schools for early career 
Believe and Prepare teachers and other early career teachers in the state. The characteristics of early career 
Believe and Prepare teachers differed from the characteristics of other early career teachers who did not 
participate in the Believe and Prepare pilot program (figure C8). For the samples of teachers who attained a Level 
1 certificate, there were no significant differences between early career Believe and Prepare teachers and other 
early career teachers (table C7). The characteristics of districts and schools in which early career Believe and 
Prepare teachers were first employed differed significantly from the characteristics of schools and districts in 
which other early career teachers were first employed (tables C8 and C9). 
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Figure C8. Characteristics (race/ethnicity, gender, and level of education) of early career Believe and Prepare 
teachers differed from those of other early career teachers in the state, 2015/16—2017/18 


White Black mOther 
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* Significant at p < .05; ** significant at p < .01. 

Note: Chi-square tests of the characteristics of early career Believe and Prepare teachers and other teachers in the state result in p values of .004 for 
race/ethnicity, .005 for gender, and .036 for level of education. 

Source: Authors’ analysis of data provided by the Louisiana Department of Education. 


Table C7. Characteristics of early career Believe and Prepare teachers and other early career teachers in the 
state who attained a Level 1 professional certificate, 2015/16—2017/18 


Teacher characteristic Early career Believe and Prepare teachers Other early career teachers 
with Level 1 certificate with Level 1 certificate 


Number Percent Number Percent 


Race/ethnicity 


Black 36 18.56 557 14.3 
White 154 79.38 3,190 81.92 
Other 4 2.06 147 3.78 
Total 194 100 3,894 100 
Highest level of education 

Bachelor’s degree 169 85.80 3110 78.67 
Education specialist 0 0 4 0.1 
Higher than master’s degree ? a 82 2.07 
Master’s degree 26 13.33 757 19.15 
Total 195 100.0 3,953 100.0 
Gender 

Female 166 84.26 3,279 82.87 
Male 31 15.74 678 17.13 
Total 197 100 3,957 100 


Note: Chi-square tests of the compositions of early career Believe and Prepare participants and other teachers in the state who attained the Level 1 
professional certificates resulted in p-values of .141 for race/ethnicity, .680 for gender, and .113 for level of education. 

a. Too low to report. 

Source: Authors’ analysis of data provided by the Louisiana Department of Education and publicly available data from the department’s Data Center webpage 
(https://www.louisianabelieves.com/resources/library/data-center) and from National Center for Education Statistics (2016, 2017, 2018, 2019). 


Table C8. Characteristics of schools in which early career Believe and Prepare teachers and other early career 
teachers in the state were first employed, 2015/16-2017/18 
Early career Other 


Believe and Prepare teachers early career teachers 
(n = 167 schools) (n = 1,322 schools) 
Slilalel-lac! Slilalel-lae! 
Yel solo) ol a¥-Te-lei kasd (os Mean deviation Median Mean deviation Median p-value 
Total enrollment 629 355 567 536 328 466 .001 
Percentage of economically disadvantaged students 76.46 17.90 79.48 73.63 18.21 76.75 .056 
Percentage of racial/ethnic minority students 65.21 26.90 66.39 59.54 30.54 59.37 .012 
Percentage of English learner students 2.27 3.64 1.05 3.15 6.04 0.99 .008 


Source: Authors’ analysis of data provided by the Louisiana Department of Education and publicly available data from the department’s Data Center webpage 
(https://www. louisianabelieves.com/resources/library/data-center). 


Table C9. Characteristics of districts in which early career Believe and Prepare teachers and other early career 
teachers in the state were first employed, 2015/16-2017/18 


Early career Other 
Believe and Prepare teachers early career teachers 
(n = 52 districts) (n = 148 districts) 
S\r-Taler-lae| Siv-laler-lae| 
District characteristic Mean deviation Median Mean deviation Median 
Total enrollment 10,167 11,553 5,962 4,873 8,762 1,302 .004 
Percentage of economically disadvantaged students 74.34 12.79 75.61 76.31 17.63 79.35 391 
Percentage of racial/ethnic minority students 54.46 22.34 52.28 63.17 28.55 61.16 .027 
Percentage of English learner students 2.24 2.64 1.40 1.56 2.33 0.58 .101 


Note: The number of districts reported by the Louisiana Department of Education varied over the study years. The data used to calculate the characteristics 
of districts for the sample of other early career teachers in the state were extracted across several years. 

Source: Authors’ analysis of data provided by the Louisiana Department of Education and publicly available data from the department’s Data Center webpage 
(https://www. louisianabelieves.com/resources/library/data-center). 


Analyses exploring the similarities between residency and employment schools and the relationship of similarities 
to retention of early career Believe and Prepare pilot participants. For Believe and Prepare pilot participants whose 
employment school in their first year of teaching was not their school of residency, neither retention nor the 
characteristics of schools of employment and residency differed significantly (tables C10 and C11). 


Table C10. Comparison of retention rates between early career Believe and Prepare teachers who gained 
employment in their school of residency and those who gained employment in another school, 2015/16- 
2017/18 


First employment school is residency school 


No (n = 102) Yes (n = 140) 


Retention Number Percent Number Percent 

In the state 92 90 123 88 .716 
In the district 75 74 109 78 531 
In the school 67 66 105 75 .152 


Source: Author’s analysis of data provided by the Louisiana Department of Education. 


Table C11. Characteristics of residency and employment schools of early career Believe and Prepare teachers 
who gained employment in schools that were not their school of residency, 2015/16-2018/19 


Residency Employment 

schools Yel afore) [5 

(n = 56) (n = 87) 
NYol afore) Meial-le-leikclaciule Mean Mean t-statistic 
Total enrollment 698 682 .20 .84 
Percentage of economically disadvantaged students 77.43 72.44 1.51 14 
Percentage of racial/ethnic minority students 67.51 60.70 1.50 14 
Percentage of English learner students 3.75 2.50 1.33 .19 


Source: Authors’ analysis of data provided by the Louisiana Department of Education and publicly available data from the department’s Data Center webpage 
(https://www. louisianabelieves.com/resources/library/data-center). 
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Characteristics of Believe and Prepare pilot participants with and without identifiers 

This section reports on comparisons of the 468 Believe and Prepare pilot participants with identifiers and the 245 
participants without identifiers. The study team had data on cohort, teacher preparation programs, subject in the 
teacher preparation program, and residency district and school identifiers for each of these two groups. 


Believe and Prepare pilot participants without identifiers were distributed across multiple teacher preparation 
program providers (see figure C1). However, four programs had a disproportionately higher number of teachers 
without identifiers: Louisiana State University-Baton Rouge, McNeese State University, Xavier University, and 
Tulane University. 


Believe and Prepare pilot participants with identifiers and those without identifiers differed by pilot cohort and 
teacher preparation program subject (tables C12 and C13). Participants without identifiers were primarily from 
cohort 3 (79 percent). There were no statistically significant differences in the characteristics of the residency 
schools and districts of Believe and Prepare pilot participants with identifiers and those without identifiers (tables 
C14 and C15). 


Table C12. Number and percentage of Believe and Prepare pilot participants with identifiers and those 
without identifiers, by cohort, 2014/15-2016/17 
VViivamlel=Vaidiiteve Without identifier 
(n = 468 participants) (eo of- Vad (oll oF 1a 9) 


(eo) ao) at Number Percent Number Percent 


Note: A chi-square test of the characteristics of the two groups resulted in a p-value of < .001, indicating that the distribution of participants across cohorts 
differed between the two groups. 
Source: Author’s analysis of data from the Louisiana Department of Education 


Table C13. Number and percentage of Believe and Prepare pilot participants with identifiers and those 
without identifiers, by teacher preparation program subject, 2014/15-2016/17 


With identifier Without identifier 
(n = 408 participants) (n = 105 participants) 
Subject Number Percent Number Percent 
English language arts 46 11 38 36 
Elementary 186 46 53 50 
Math 35 9 0 0 
Other 30 7 3 3 
Science 27 7 0 0 
Social studies 18 4 2 2 
Special education 38 9 9 9 
Subject specific—other 28 7 0 0 


Note: Data on teacher preparation program subject were missing in 140 records without identifiers and 60 records with identifiers. A chi-square test of the 
characteristics of the two groups resulted in p-value of < .001, indicating that the distribution of subjects differed between the two groups. 
Source: Author’s analysis of data from the Louisiana Department of Education. 
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Table C14. Characteristics of schools of residency for Believe and Prepare participants with identifiers and 
those without identifiers, 2014/15-2016/17 


With identifier Without identifier 
(eae kt: Yel aoe) 9) (n = 89 schools) 
Simlatel-]ae} Standard 
Characteristic of residency schools Mean deviation Mean deviation p-value 
Total enrollment 653 393 724 439 .20 
Percentage of economically disadvantaged students 77.64 16.81 77.00 18.38 78 
Percentage of racial/ethnic minority students 64.53 28.52 68.37 28.91 30 
Percentage of English learner students 2.45 4.23 3.30 5.20 18 


Note: The p-values indicate the level of statistical significance of the difference between the two groups for each school characteristic. 
Source: Author’s analysis of data from the Louisiana Department of Education and publicly available data from the department’s Data Center webpage 
(https://www.louisianabelieves.com/resources/library/data-center). 


Table C15. Characteristics of districts of residency for Believe and Prepare participants with identifiers and 
those without identifiers, 2014/15-2016/17 


With identifier Without identifier 
(n = 38 districts) (n = 28 districts) 
Siilalel-lae! Silalel-lac! 

Characteristic of residency district Mean deviation Mean deviation p-value 
Total enrollment 10,837 11,296 11,646 10,995 77 
Percentage of economically disadvantaged students 75.05 12.95 74.21 14.60 81 
Percentage of racial/ethnic minority students 54.91 21.19 49.86 21.81 35 
Percentage of English learner students 2.14 2.22 2.75 2.68 33 


Note: The p-values indicate the level of statistical significance of the difference between the two groups for each district characteristic. 
Source: Author’s analysis of data from the Louisiana Department of Education and publicly available data from the Louisiana Department of Education’s Data 


Center webpage (https://www. louisianabelieves.com/resources/library/data-center). 


Appendix D. Other analyses 


This appendix provides findings from the supplemental analysis that explored the associations between the 
attainment of a Level 1 professional certificate and characteristics of schools of residency (research question 2a) 
and the associations between entering teaching and characteristics of schools of residency (research question 3a) 
when the 245 Believe and Prepare pilot participants without identifiers were included in the analysis. The 
supplemental analyses assumed that the 245 participants did not attain certification and did not teach in Louisiana 
and that they were all early in their career. The results are largely consistent with those from the main analysis. 


Findings on certification 

Among the sample of 581 early career Believe and Prepare pilot participants that included the 245 records without 
identifiers, 268 participants attained a Level 1 professional certificate between 2015/16 and 2017/18.7° Only one 
characteristic of the schools of residency was associated with the probability of early career Believe and Prepare 
pilot participants attaining a Level 1 professional certificate, after other school characteristics were controlled for. 
Participants who served their residency in a charter school had a lower probability (by 22 percentage points) of 
attaining the certificate than those who served their residency in a noncharter school. For comparison, analyses 
based on the main analytic sample (presented in the report) found differences in the probability of certification 
for participants who served their residency in a primary school as well as for participants who served their 
residency in a charter school. Participants who served a residency in a primary school had a higher probability (by 
11 percentage points) of attaining the certificate than those who served their residency in a nonprimary school, 
and participants who served a residency in a charter school had a lower probability (by 24 percentage points) of 
attaining the certificate than those who served a residency in a noncharter school. 


Findings on employment 


Among the sample of 581 early career Believe and Prepare pilot participants that included the 245 records without 
identifiers, the 58 participants who were already teaching in Louisiana in 2014/15 were not included in the analysis 
for employment outcomes. Among the remaining 523 participants, 255 entered teaching in a Louisiana public 
school between 2015/16 and 2018/19.7! Teachers who attained a Level 1 professional certificate were more likely 
to enter teaching in the state than teachers who had not attained the certificate (by 64 percentage points), after 
school characteristics were adjusted for. None of the characteristics of residency schools were associated with 
entering teaching. For comparison, analyses based on the main analytic sample found that none of the predictors 
examined—neither attainment of a Level 1 professional certificate nor any of the school characteristics—were 
associated with entering teaching. 


10 Participants who attained a Level 1 professional certificate between 2015/16 and 2017/18 made up 46 percent of the sample for the 
supplemental analysis that included the 245 participants without identifiers and 80 percent of the sample for the main analysis (a total of 
336 participants) that excluded those 245 participants. 

1 Participants who entered teaching accounted for 49 percent of the sample for the supplemental analysis that included the 245 
participants without identifiers, and 92 percent of the sample for the main analysis (a total of 278 participants) that excluded those 245 
participants. 
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